Jerry Alan Fails' Thesis Proposal

IMAGE PROCESSING WITH CRAYONS Jerry Alan Fails Department of Computer Science Masters of Science As computers become faster, smaller, and less expensive, ubiquitous computing, the distribution of computers into everyday spaces, grows at an ever-increasing rate. One area of particular interest is perceptual user interfaces (PUIs). PUI systems use sensing-devices for input, which are coupled with a processor to execute commands based on the detected inputs. Each PUI system requires a classifier to classify the inputs from the sensing device, so that the corresponding command can be executed. This thesis explores camera-based PUIs. Camera-based PUIs require a visual classifier to classify the input images. The creation of such classifiers is costly in terms of time and resources. By creating a tool to facilitate classifier creation, PUI development is dramatically simplified. This thesis concentrates on the issues and implementation details of building such a tool. The tool created in this project is Image Processing with Crayons (Crayons). Crayons endeavors to meet the fast and focused user interface principle, which is that a user interface should allow its user to quickly complete his/her task without being distracted from it. Crayons’ three components, the user interface, machine learning, and image processing (features), all strive to meet this principle. The user interface overcomes the problem of presenting a large amount of data by overlaying coinciding data. This facilitates user focus by having one main interactive control. Interactive machine learning and modified decision trees are used to attain the speed and feature selection necessary to help meet the fast and focused UI principle. Crayons uses color region features. These features are calculated using integral images that enable rectangular regional summations in constant time. The efficiency of using texture features is compared to that of color region features, showing the use of color region features in Crayons to be more effective. All of these components collectively combine to meet the fast and focused UI principle, allowing Crayons to reach its overarching goal — to create image classifiers in minutes not months.